1. Field of the Invention
The present invention relates to a device for detecting the line of sight of the observer, utilizing an image of the eyeball thereof.
2. Related Background Art
There have been proposed various devices for detecting the position in the viewing field, observed by the observer, or so-called line of sight thereof, such as the eye camera.
For example, the Japanese Patent Laid-Open Application No. 1-274736 discloses a device for projecting a parallel light beam from a light source to the frontal part of the eyeball of the observer, and determining the line of sight by means of the corneal reflected image and the image position of the pupil, formed by the light reflected from the cornea.
FIG. 17 shows the principle of detection of the line of sight, and FIGS. 18A and 18B are respectively a view showing the image of eyeball projected onto an image sensor shown in FIG. 17 and a chart showing output intensity from the image sensor 14.
In the following there will be given an explanation on the method of detecting the line of sight, making reference to FIGS. 17, 18A and 18B. Infrared light-emitting diodes 13a, 13b are positioned substantially symmetrically with respect to the optical axis (i) of a light-receiving lens 12, or to the Z-axis, and respectively illuminate the eyeball of the observer in a diffuse manner.
The infrared light emitted from the LED 13b illuminates the cornea 16 of the eyeball 15. A corneal reflected image d, formed by a part of the infrared light reflected at the surface of the cornea 16, is refocused by the light-receiving lens 12 at a position d' on an image sensor 14.
Similarly the infrared light emitted from the LED 13a illuminates the cornea 16 of the eyeball. A corneal reflected image e, formed by a part of the infrared light reflected at the surface of the cornea 16 is refocused by the light-receiving lens 12 at a position e' on the image sensor 14.
Also the light beams from end positions a, b of the iris 17 are condensed by the light-receiving lens 12 to form images of said end positions a, b at positions a', b' on the image sensor 14. When the rotation angle .theta. of the optical axis (ii) of the eyeball 15 is small with respect to the optical axis (i) of the light-receiving lens 12, the coordinate Xc of the center of the pupil 19 can be represented by the coordinates xa, xb of said end positions a, b of the iris 17 as follows: EQU Xc=(Xa+Xb)/2.
Also, since the Z-coordinate of the center of the corneal reflected images d, e coincides with the Z-coordinate Zo of the center O of curvature of the cornea 16, the rotation angle .theta. of the optical axis (ii) of the eyeball approximately satisfies a relation: EQU (A1 * L.sub.OC) * sin.theta..congruent.Xc-(Xd+Xe)/2 (1)
wherein Xd, Xe are X-coordinates of the positions d, e where the corneal reflected images are generated, L.sub.OC is a standard distance from the center O of curvature of the cornea 16 to the center C of the pupil 19, and A1 is a coefficient representing individual fluctuation on said distance L.sub.OC. Consequently, in a sight line calculating device, the rotation angle .theta. of the optical axis (ii) of the eyeball can be determined by detecting the positions of feature points (corneal reflected images d, e and end positions a, b of the iris) projected on the image sensor, as shown in FIGS. 18A and 18B. In this operation, the equation (1) is re-written as: EQU .beta.(A1 * L.sub.OC) * sin.theta..congruent.(Xa'+Xb')/2-(Xd'+Xe')/2 (2)
wherein .beta. stands for a magnification determined by the position of the eyeball with respect to the light-receiving lens 12, and is practically determined as a function of the distance .vertline.Xd'Xe'.vertline. of the corneal reflected images. Also the rotation angle .theta. of the eyeball 15 is re-written as: EQU .theta..congruent.arcsin{(Xc'-Xf')/.beta./(A1 * L.sub.OC)} (3)
wherein EQU Xc'.congruent.(Xa'+Xb')/2 EQU Xf'.congruent.(Xd'+Xe')/2.
Since the optical axis (ii) of the eyeball of the observer does not coincide with the line of sight, the horizontal line of sight .theta.H of the observer can be determined by an angular correction .delta. between the optical axis of the eyeball and the line of sight, once the rotation angle .theta. of the optical axis (ii) of the eyeball in the horizontal direction is calculated. Taking a coefficient B1 for the individual fluctuation for the correction angle .delta. between the optical axis (ii) of the eyeball and the line of sight, the line of sight .theta.H of the observer in the horizontal direction can be given by: EQU .theta.H=.theta..+-.(B1 * .delta.) (4)
wherein the sign .+-. is + or - respectively if the observer looks at the device with the left eye or the right eye, when the rotation angle to the right with respect to the observer is taken as positive.
FIG. 17 shows a case of the rotation of the eyeball of the observer in the Z-X plane (for example horizontal plane), but the detection is similarly possible also in case of rotation of the eyeball in the Z-Y plane (for example vertical plane). However, since the vertical component of the line of sight of the observer coincides with the vertical component .theta.' of the optical axis of the eyeball, the line of sight .theta.V in the vertical direction is represented by: EQU .theta.V=.theta.'.
Based on the sight line data .theta.H, .theta.V, the coordinates (Xn, Yn) looked at by the observer on the focus screen in a view finder field is given by: ##EQU1## wherein m is a constant determined by the finder optical system of the camera.
The coefficients A1, B1 for correcting the individual fluctuation of the eyeball of the observer can be determined by letting the observer watch an index provided in a predetermined position in the view finder and matching the position of the watched point calculated according to the equation (5) with the position of said index.
The calculation for determining the line of sight of the observer and the watched point is executed by the software of a microcomputer of the sight line processing device, according to the foregoing equations.
Since the coefficient for correcting the individual difference in the line of sight generally corresponds to the horizontal rotation of the eyeball of the observer, the two indexes provided in the view finder are positioned in the horizontal direction to the observer.
After the determination of said coefficient for correcting the individual difference in the line of sight, the position, on the focusing screen, of the line of sight of the observer looking at the view finder is calculated according to the equation (5), and thus obtained information on the line of sight is utilized for focusing control of the phototaking lens or for exposure control.
For detecting the corneal reflected image (Purkinje's image) or the ends of the pupil, it is necessary that an image signal of the eyeball image can be obtained in constantly stable manner. For this purpose there is usually adopted automatic gain control, but, in case of a sight line detecting device often used outdoors, such as the one incorporated in the camera, the signal level of the eyeball image is considerably unstable by the external noises. Thus, even under the application of automatic gain control, there have been encountered drawbacks of erroneous detection of the corneal reflected image (hereinafter called Purkinje's image) as an external ghost, or of deficient output level at the ends of the pupil, rendering the sight line detection impossible or generating a significant error in the detection. Also since the Purkinje's image and the end portions of the pupil are strongly affected by the state of external light or by the position of the eye-ball of the observer, an algorithm enabling constant detection will give excessive burden to the microcomputer, and, in fact, it has been extremely difficult to prepare such an algorithm.
The present applicant already proposed, in the Japanese Patent Laid-Open Application No. 2-213322, in forming a Purkinje's image by projecting a parallel light beam onto the eyeball, to alter the projecting direction of said parallel light beam if the Purkinje's image overlaps with the pupil edge image.